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CATEGORY  ANALYSIS  OF  THE  MARINE  CORPS 
COMBAT  READINESS  EVALUATION  SYSTEM  (MCCRES) 

by 

S.  Zacks,  W.  H.  Marlow  and  Zeev  Barzily 


1.  Introduction 

The  Marine  Corps  Combat  Readiness  Evaluation  System  (MCCRES)  and 
various  analyses  have  been  discussed  recently  in  a  series  of  papers  by 
Barzily,  CataJogne  and  Marlow  [2],  Zacks  and  Marlow  [7]  and  Barzily, 
Marlow,  and  Zacks  [8],  In  the  present  paper  we  conduct  statistical 
analysis  of  MCCRES  from  the  point  of  view  of  the  ten  categories 
introduced  by  Barzily  [1],  As  in  the  previous  work  we  restrict  our 
present  attention  to  Volume  II  of  MCCRES,  which  deals  with  infantry 
battalions  and  contains  800  different  requirements.  The  category 
approach  of  Barzily  classifies  all  requirements  into  the  following 
ten  categories. 

] .  REPORTING  to  higher  levels  of  command 

2.  PREPARING  for  operation 

3.  COMMUNICATING 

4.  PERFORMING  as  Marines 

5.  DELIVERING  supporting  fire 

6.  PLANNING  of  operations 

7.  CONFORMING  to  doctrine 
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8.  EXECUTING  operations 

9.  PROVIDING  combat  service  support 

10.  SUPERVISING  required  actions  of  individual  Marines 

In  a  given  evaluation  an  infantry  battalion  is  evaluated  on  a  subset  of 
the  800  requirements.  That  is,  due  to  time,  budget,  and  other  constraints, 
not  all  800  requirements  are  applicable.  The  battalion  is  given  a  score 
of  "Yes"  in  case  a  requirement  is  fulfilled  satisfactorily,  and  a  score  of 
"No"  otherwise.  The  basic  assumption  in  our  category  approach  is  that  the 
probability  of  "Yes"  for  a  given  unit  is  the  same  for  all  the  applicable 
requirements  of  the  same  category  (see  Appendix  1).  These  probabilities 
may  change  from  one  category  to  another.  The  vector  of  the  ten  probabili¬ 
ties,  corresponding  to  the  ten  categories,  is  a  parametric  vector  which 
characterizes  the  unit  at  that  particular  evaluation.  Different  units 
usually  have  different  characterizing  parametric  vectors  and  the  same  unit 
may  have  different  characterizing  parametric  vectors  in  different  evalua¬ 
tions.  In  addition  to  the  parametric  vector  of  "Yes"  probabilities,  there 
are  several  other  interesting  statistical  features  of  the  data.  The  pro¬ 
portion  of  "Yes's"  among  the  applicable  requirements  of  different  categories 
in  the  same  evaliation  are  not  independent.  It  is  interesting  to  study  the 
correlation  structure  between  these  categories.  In  the  present  study  we 
present  statistical  lodels  and  develop  procedures  designed  to  estimate 
interesting  model  parameters.  We  base  the  analysis  on  the  results  of 
N  *  27  evaluations.  The  statistical  methods  developed  here  have  a  wide 
range  of  possible  applications  for  the  analysis  of  categorical  data  in 
which  a  different  number  of  independent  Bernoulli  trials  are  performed  in 
each  category,  but  the  proportions  of  successes  in  different  categories 
are  dependent  variables. 

2.  The  Structure  of  the  Data  and  the  Statistical  Model 

Let  N  designate  the  number  of  evaluations,  N  =  27,  and  k  designate 
the  number  of  categories,  k  *  10  .  Let  X„  ,  i=l,...,N  ,  j=l,...,k 

designate  the  number  of  "Yes's"  among  the  applicable  requirements  of  the 
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j-th  category  in  the  i-th  evaluation.  Let  M  ,  i=l,...,N  ,  j=l,...,k 

designate  the  corresponding  number  of  applicable  requirements.  In  Table 
2.1  we  present  the  values  of  X„  and  which  have  been  compiled  from 

MCCRES  data  files.  All  the  analyses  in  the  present  paper  are  based  on  the 
data  in  this  table. 

Let  X.  (i=l,...,N)  designate  the  vector  of  k  X  -  values  corre¬ 
al 

sponding  to  evaluation  i  ,  and  M.  the  vectors  of  corresponding  M  -  values. 

Oil 

The  fundamental  assumptions  of  the  statistical  model  are: 


A.l  X^  ,  are  independent  random  vectors; 

A. 2  The  marginal  distribution  of  X„  is  binomial, 

B(M_  ,  0„)  ,  for  all  i=l . N  ,  and  j=l . k  . 

0 _  designates  the  probability  of  "Yes"  for  all 
the  requirements  in  evaluation  i  and  category  j  . 


In  Appendix  1  we  test  a  null  hypothesis  which  is  based  on  assumption 
A. 2.  This  hypothesis  is  accepted  by  the  data  in  such  a  way  to  provide 
substantial  empirical  verification. 


The  multivariate  distributions  of  X,  ,  i=l,...,N  ,  are  specified 
indirectly  by  considering  first  the  following  transformations.  Let 


Y.  . 


2  sin 


i=l, 

j=l* 


,N 


(2.1) 


These  transformations  are  known  as  variance  stabilizing  transformations  for 
binomial  random  variables  (see  Johnson  and  Kotz  [3;  p.  65],  who  suggest 
the  transformation  Y  =  2  sin  ^  *  where  0  =  (x  +  3/8)/(M  +  3/4)  . 

The  asymptotic  properties  of  (2.1)  and  of  this  one  are  the  same). 
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Total  number  of  satisfied  and  applicable  requirements 
by  category  and  evaluation* 


EVAL/CAT. 

1 

2 

3 

4 

1 

62 

44 

40 

14 

67 

54 

40 

23 

2 

79 

70 

42 

18 

80 

76 

44 

24 

3 

41 

19 

26 

12 

42 

26 

28 

21 

4 

60 

33 

25 

20 

62 

36 

31 

24 

5 

35 

34 

35 

21 

52 

44 

37 

24 

6 

52 

29 

20 

23 

55 

29 

29 

24 

7 

44 

41 

32 

15 

47 

46 

34 

21 

8 

29 

18 

28 

12 

60 

48 

32 

22 

9 

64 

41 

36 

25 

71 

48 

37 

26 

10 

64 

48 

35 

19 

70 

55 

38 

24 

11 

15 

4 

23 

18 

15 

4 

23 

18 

12 

12 

5 

14 

16 

14 

8 

17 

16 

13 

11 

11 

5 

6 

11 

11 

6 

9 

14 

54 

31 

23 

17 

54 

35 

27 

25 

15 

41 

21 

13 

16 

42 

23 

17 

23 

16 

49 

35 

29 

13 

54 

38 

32 

23 

17 

44 

29 

18 

16 

47 

30 

21 

23 

18 

69 

40 

35 

26 

69 

41 

36 

28 

19 

78 

43 

38 

27 

79 

51 

38 

27 

20 

48 

26 

25 

13 

54 

31 

29 

24 

21 

18 

5 

14 

4 

24 

11 

21 

17 

22 

71 

35 

29 

21 

73 

37 

34 

25 

23 

71 

41 

33 

26 

73 

41 

36 

28 

24 

47 

30 

28 

22 

49 

31 

30 

25 

25 

44 

32 

29 

16 

48 

34 

32 

25 

26 

34 

29 

24 

21 

37 

32 

27 

23 

27 

100 

58 

32 

27 

101 

58 

38 

29 

5 

6 

7 

8 

9 

10 

55 

100 

50 

53 

24 

13 

69 

110 

63 

73 

32 

17 

67 

133 

65 

97 

31 

17 

72 

140 

73 

100 

37 

20 

45 

38 

44 

47 

15 

12 

56 

45 

53 

54 

16 

14 

53 

68 

49 

68 

18 

16 

55 

71 

54 

72 

19 

16 

52 

93 

46 

46 

18 

12 

54 

99 

54 

67 

21 

14 

57 

60 

59 

62 

18 

18 

63 

63 

61 

67 

18 

18 

63 

90 

47 

50 

30 

13 

64 

93 

51 

55 

31 

17 

48 

40 

34 

65 

7 

9 

63 

81 

57 

87 

14 

15 

64 

78 

44 

93 

16 

18 

64 

90 

57 

104 

18 

18 

60 

99 

45 

67 

25 

17 

67 

119 

61 

77 

30 

17 

56 

15 

30 

17 

13 

11 

56 

15 

31 

17 

13 

12 

35 

10 

16 

12 

8 

7 

54 

16 

30 

18 

10 

10 

6 

15 

8 

36 

2 

5 

6 

16 

10 

37 

2 

5 

57 

55 

49 

62 

16 

13 

59 

62 

57 

70 

17 

18 

4 

36 

33 

44 

13 

11 

4 

36 

35 

47 

13 

16 

59 

73 

59 

63 

18 

12 

59 

74 

63 

70 

18 

16 

9 

52 

44 

54 

18 

13 

9 

52 

45 

55 

19 

13 

62 

76 

66 

87 

22 

14 

63 

78 

68 

90 

22 

16 

64 

90 

60 

92 

19 

17 

65 

98 

71 

108 

20 

17 

58 

49 

38 

56 

16 

17 

61 

61 

56 

67 

18 

18 

41 

14 

22 

12 

12 

9 

56 

22 

36 

28 

14 

15 

52 

85 

45 

85 

24 

19 

66 

92 

54 

90 

24 

20 

63 

87 

63 

93 

21 

15 

64 

88 

68 

97 

23 

16 

55 

61 

52 

49 

13 

16 

57 

63 

57 

58 

15 

17 

31 

56 

50 

47 

15 

16 

61 

64 

58 

58 

16 

18 

52 

48 

32 

42 

15 

14 

58 

56 

40 

54 

15 

15 

69 

121 

75 

136 

26 

18 

71 

122 

78 

138 

28 

19 

’•The  upper  number  in  each  cell  represents  the  total  number  of  satisfied 
requirements;  the  lower  number  represents  the  total  number  of  applicable 


requirements. 
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For  large  values  of  M. .  the  random  variables  Y..  are  distributed 

ij  ij 

approximately  normally,  with  mean  .  =  2  sin  ^  v^T  and  variance 

I 

1/M..  .  Let  Y.  =  (Y.,  ,...,Y.,)  be  the  k-variate  vector  of  the 
ij  -vi  xl  lk 

transformed  X-values  of  the  i-th  evaluations.  According  to  assumption 
A.l,  Y^,...,Y^  are  independent  random  vectors.  We  introduce  here  a  third 

assumption  concerning  the  multivariate  distribution  of  Y.  (i=l,...,N)  . 

A. 3  The  random  vectors  Y.  have  multivariate  normal 

'XjI 


distributions  with  means  r|.  =  (h 

'\A 

covariance  matrices  t.  =  (t.  ..,)  , 

Ti  ri*JJ 


• ,nik)  and 


where 


x.  ...  - 


JJ 


1 

if 

j=j' 

= 

M.  . 

13 

p.  . ,  /  fn.  .m. . ,  , 

JJ  /  V  IJ  IJ 

if 

j+j'  • 

correlation  between  Y.. 

ij 

and 

V 

(2.2) 


It  is  assumed  that  the  correlations  p...  are  the  same  for  all  evaluations. 

JJ 

In  Appendix  2  we  show  that  Pj j .  are  asymptotically  equivalent  to  the 

A 

correlations  between  the  proportions  of  Yes's,  0..  =  X../M..  . 

ij  ij  ij 


3.  The  Bayesian  Framework 

The  Bayesian  model  assumes  that  the  characterizing  parametric  vectors 
0  of  the  different  evaluations  vary  according  to  a  (prior)  distribution 

Oii 

of  all  possible  evaluations.  This  is  a  model  of  a  superpopulation  which 
makes  it  possible  to  relate  the  information  obtained  from  all  evaluations 
in  a  consistent  manner .  Following  the  model  developed  in  the  previous 
section,  we  present  here  the  Bayesian  model  in  terms  of  the  distributions 
of  Y.  (i=l,..., N)  . 
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We  assume,  that  the  mean  vectors  p .  of  the  distributions  of 

Y  (i=l,...,N)  have  a  prior  normal  distribution  with  mean  vector  p  and 
1 

diagonal  covariance  matrix  (D)  .  The  overall  distribution  of  Y  .  , 

ij 

where  p^.  ate  treated  as  random  variables,  is  called  a  predictive 

distribution.  According  to  the  above  assumption,  the  predictive  distribu¬ 
tion  of  each  Y.  is  normal  with  the  same  mean,  n  ,  and  covariance  matrix 

'v1 

+  (D)  (see  Zacks  [6;  p.  296]).  The  predictive  marginal  distributions 
of  the  components  Y_  are  normal  with  means  p  ( j  =1 . k)  and  variances 

2  _  1  2 

T..  -  Tj —  +  D  (i=l,...,N  ,  j=l,...,k)  .  Since  we  assumed  that  the 
J  ij  J 

correlations  ,  are  the  same  for  all  evaluations,  we  will  treat  the 

vectors  n  ,  and  the  matrices  (D)  and  (R)  =  (p..,  ;  j,j'  =*  l,...,k)  as 
^  J  J 

unknown  but  fixed  parameters.  We  remark  that  the  predictive  marginal 

distributions  of  Y..  resemble  the  distributions  of  the  observations  in 

ij 

Model  II  of  Analysis  of  Variance  (see  Zacks  [6;  p.  79]),  in  which  the 
components  of  variance  are  1/M„  (known)  and  (unknown).  We 

conclude  this  section  by  specifying  the  posterior  distributions  of  p 

given  Y  .  According  to  the  Bayesian  model,  the  posterior  distribution 

of  p . .  given  Y . ,  is  normal  with  mean 
ij  i] 


E  KjiV  ‘  ("j +  DjVij)/a  +  “/ij*  ■ 


(3.1) 


and  variance 


v  (n^iv  -ya  +  w 


(3.2) 


Notice  that  the  Bayes  estimator  of  p  for  a  squared-error  loss  is  given 
by  (3.1),  with  a  Bayes  risk  given  by  (3.2).  If  p^  and  are  un¬ 

specified,  one  can  estimate  them  from  the  data,  to  obtain  empirical 
Bayes  estimators  of  the  characterizing  parameters  (see  Zacks  [6;  p.  321]). 
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4 .  Bayesian  Confidence  Inrervals  for  the  Characterizing 
Parameters 

Bayesian  confidence  intervals,  at  level  1-a  ,  for  specified  para¬ 
meters,  are  defined  as  the  intervals  in  the  parameter  space  whose  posterior 
coverage  probabilities,  given  the  sample  data,  are  at  least  1-a  . 
According  to  the  model  of  Section  3,  the  posterior  coverage  probability 
of  the  intervals 


H.  +  D.M. ,Y, .  D 

_ 1  Jj-Ll  +  7  — J 

1  +  I..M.  .  -  1-a/ 2  1  +  M.  .D. 

J  1J  J 

is  (1-a)  ,  where  is  the  y-th  fractile  of  the  standard  normal  dis¬ 

tribution.  We  determine  for  each  category  empirical  estimates  of  the 
prior  parameters  r)j  and  and,  by  substituting  these  estimates  in 

(4.1),  we  obtain  empirical  Bayes  confidence  intervals  for  (i=l,...,N) 

The  substitution  of  estimates  of  n.  and  D.  for  the  correct  values  will 

i  J 

generally  effect  the  coverage  probabilities.  The  estimates  described  below 
are  consistent,  and  we  expect  that,  for  large  values  of  N  ,  the  actual 
coverage  will  be  close  to  the  nominal  one.  In  order  to  provide  conserva¬ 
tive  intervals,  we  have  used  in  each  category,  the  Bonferroni  method  of 
simultaneous  confidence  intervals,  for  all  the  N  =  27  evaluations  (see 
Miller  [4]).  For  this  purpose  we  replaced  a  in  (4.1)  by  a'  =  a/N  . 

Applying  the  transformation 

0  =  sin2(n/2)  ,  (4.2) 

which  is  strictly  increasing  for  0  <  n  <  tt  ,  we  can  obtain  from  the 

n-  confidence  intervals  0  -  confidence  intervals.  These  intervals,  for 
the  ten  categories  and  twenty  seven  evaluations  with  a  =  .05  ,  are 
presented  in  Table  4.1. 
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Table  4.1 


CONFIDENCE  INTERVALS  FOR  CAT.  C1-C5 


12  3  4 


EVAL 

L 

U 

L 

U 

L 

U 

L 

U 

L 

1 

0.802 

0.987 

0.648 

0.942 

0.849 

0.999 

0.366 

0.882 

0.661 

2 

0.902 

1.000 

0.794 

0.982 

0.802 

0.988 

0.481 

0.942 

0.809 

3 

0.834 

1.000 

0.519 

0.942 

0.751 

0.984 

0.333 

0.872 

0.655 

4 

0.853 

0.999 

0.723 

0.991 

0.683 

0.956 

0,556 

0.972 

0.834 

5 

0.548 

0.881 

0.592 

0.931 

0.782 

0.987 

0.598 

0.984 

0.832 

6 

0.814 

0.995 

0.820 

1.000 

0.625 

0.931 

0.699 

1.000 

0.769 

7 

0.791 

0.995 

0.717 

0.980 

0.772 

0.986 

0.437 

0.935 

0.880 

8 

0.397 

0.749 

0.269 

0.668 

0.723 

0.972 

0.318 

0.855 

0.622 

9 

0.778 

0.977 

0.680 

0.964 

0.807 

0.993 

0.717 

1.000 

0.913 

10 

0.792 

0.983 

0.711 

0.968 

0.763 

0.981 

0.517 

0.958 

0.763 

11 

0.755 

1.000 

0.509 

1.000 

0.794 

0.997 

0.714 

1.000 

0.902 

12 

0.634 

0.997 

0.407 

0.982 

0.679 

0.973 

0.6E8 

1.000 

0.514 

13 

0.716 

1.000 

0.659 

1.000 

0.668 

0.986 

0.328 

0.969 

0.621 

14 

0.900 

1.000 

0.685 

0.984 

0.704 

0.969 

0,427 

0.909 

0.843 

15 

0.834 

1.000 

0.669 

0.996 

0.658 

0.966 

0.432 

0.922 

0.582 

16 

0.768 

0.985 

0.734 

0.992 

0.743 

0.979 

0.335 

0.860 

0.906 

17 

0.791 

0.995 

0.767 

1.000 

0.698 

0.975 

0.432 

0.922 

0.669 

18 

0.920 

1.000 

0.817 

1.000 

0.804 

0.993 

0.682 

0.995 

0.878 

19 

0.901 

1.000 

0.673 

0.957 

0.844 

0.999 

0.793 

1.000 

0.881 

20 

0.747 

0.979 

0.626 

0.972 

0.712 

0.970 

0.321 

0.843 

0.824 

21 

0.586 

0.965 

0.331 

0.929 

0.621 

0.943 

0.128 

0.711 

0.589 

22 

0.871 

0.999 

0.762 

0.997 

0.712 

0.965 

0.568 

0.973 

0.649 

23 

0.871 

0.999 

0.865 

1.000 

0.757 

0.980 

0.682 

0.995 

0.880 

24 

0.823 

0.998 

0.772 

1.000 

0.758 

0.985 

0.609 

0.985 

0.839 

25 

0.770 

0.990 

0.747 

0.997 

0.743 

0.979 

0.396 

0.890 

0.401 

26 

0.753 

0.994 

0.699 

0.991 

0.724 

0.977 

0.634 

0.994 

0.754 

27 

0.921 

1.000 

0.901 

1.000 

0.709 

0.960 

0.691 

0.995 

0.866 

CONFIDENCE 

INTERVALS 

FOR  CAT 

C6-C10 

1 

0.807 

0.971 

0.642 

0.921 

0.586 

0.876 

0.771 

0.942 

0.611 

2 

0.872 

0.988 

0.752 

0.966 

0.881 

0.997 

0.789 

0.950 

0.657 

3 

0.675 

0.966 

0.665 

0.946 

0.710 

0.967 

0.804 

0.965 

0.644 

4 

0.844 

0.996 

0.747 

0.979 

0.821 

0.991 

0.808 

0.966 

0.736 

5 

0.840 

0.98/ 

0.727 

0.971 

0.545 

0.857 

0.793 

0.958 

0.644 

6 

0.827 

0.996 

0.835 

0.997 

0.791 

0.985 

0.819 

0.971 

0.749 

7 

0.877 

0.997 

0.759 

0.984 

0.755 

0.982 

0.825 

0.970 

0.611 

8 

0.385 

0.704 

0.465 

0.812 

0.616 

0.880 

0.759 

0.944 

0.549 

9 

0.746 

0.954 

0.615 

0.914 

0.783 

0.963 

0.797 

0.961 

0.749 

10 

0.723 

0.924 

0.588 

0.892 

0.736 

0.957 

0.789 

0.952 

0.743 

11 

0.733 

1.000 

0.765 

0.999 

0.737 

1.000 

0.811 

0.970 

0.662 

12 

0.451 

0.950 

0.406 

0.843 

0.470 

0.945 

0.788 

0.960 

0.586 

13 

0.671 

1.000 

0.523 

0.986 

0.801 

1.000 

0.794 

0.966 

0.640 

14 

0.744 

0.974 

0.700 

0.957 

0.746 

0.967 

0.805 

0.965 

0.594 

15 

0.856 

1.000 

0.747 

0.994 

0.775 

0.993 

0.811 

0.970 

0.580 

16 

0.89' 

1.000 

0.796 

0.987 

0.763 

0.974 

0.819 

0.971 

0.604 

17 

0.895 

1.000 

0.823 

0.999 

0.856 

1.000 

0.808 

0.966 

0.714 

18 

0.876 

0.999 

0.850 

0.997 

0.869 

0.996 

0.826 

0.973 

0.658 

19 

0.812 

0.978 

0.  701 

0.944 

0.737 

0.937 

0.809 

0.966 

0.743 

20 

0.654 

0.934 

0.532 

0.863 

0.689 

0.944 

0.797 

0.961 

0.705 

21 

0.464 

0.928 

0.463 

0.863 

0.323 

0.797 

0.792 

0.961 

0.549 

22 

0.815 

0.982 

0.670 

0.947 

0.837 

0.989 

0.829 

0.974 

0.717 

23 

0.909 

1.000 

0.789 

0.983 

0.862 

0.993 

0.804 

0.963 

0.691 

24 

0.851 

0.999 

0.758 

0.980 

0.688 

0.954 

0.793 

0.961 

0.698 

25 

0.733 

0.968 

0.704 

0.958 

0.653 

0.937 

0.804 

0.965 

0.672 

26 

0.704 

0.964 

0.617 

0.943 

0.617 

0.923 

0.814 

0.970 

0.684 

27 

0.933 

1.000 

0.847 

0.994 

0.924 

0.999 

0.810 

0.964 

0.711 

U 

0.926 

0.988 

0.940 

0.999 

0.999 

0.980 

1.000 

0.912 

1.000 

0.975 

1.000 

0.858 

1.000 

0.999 

1.000 

1.000 

1.000 

1.000 

1.000 

0.996 

0.903 

0.924 

1.000 

0.999 

0.754 

0.979 

0.999 


0.962 

0.973 

0.979 

0.997 

0.979 

0.997 

0.962 

0.940 

0.997 

0.997 

0.987 

0.968 

0.993 

0.952 

0.951 

0.961 

0.996 

0.980 

0.997 

0.989 

0.940 

0.990 

0.989 

0.989 

0.981 

0.988 

0.990 
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The  empirical  estimates  for  n j  and  applied  here  are: 


nj,N  =  N  Yij  ' 


(4.3) 


1  ?  V2  "2  o"1 

N  ij  "  nj,N  Hj  *N  ’ 


(A. 4) 


where  [a]  =  max  (a,0)  ,  and  H.  „  is  the  harmonic  mean  of  M. .  ,  i.e., 

J,N  ij 

-1  1  N  1 

H.  .  =  — — -  Z  — -  .  Notice  that  n.  is  a  strongly  consistent 

j,N  N  i=1  M  j,N  e  7 

estimator  of  n.  (approaches  n.  a.s.  as  N  -*•  °°)  .  Assuming  that  H. 

J  3  J  >  N 

A 

converges  to  a  limit  as  N  ^  a>  ,  D.  converges  to  D.  as  N  grows, 

J  9  N  j 

A 

The  values  of  n.  „  and  D.  „  obtained  in  this  manner  are  given  in 
J.N  J.N 

Table  4.2. 


Table  4.2 

Empirical  Estimates 

of  r|.  and  D. 

J  3 


J 

"J.H 

A 

°j.N 

1 

2.5851 

0.0755 

2 

2.4125 

0.1058 

3 

2.4494 

0.0297 

4 

2.1691 

0.1464 

5 

2.5379 

0.0871 

6 

2.5185 

0.0994 

7 

2.3468 

0.0751 

8 

2.4176 

0.0917 

9 

2.4886 

0.0086 

10 

2.3883 

0.0388 

5.  Estimating  the  Correlations 


The  marginal  bivariate  predictive  distribution  of  Y  and  t  , 


for  j  “f  j '  is  normal  wilh  p.d.f. 


f(Yij>  Yij';nj  ’  r'j'  ’  Pjj'  *  Dj  ’  Dj'}  ' 


M. 


2TT 


JJL 


"ii' 


L/2 


+  D.M. .  /  ll  +  D.M. . , 
J  1J/  \  J  iJ 


exp 


(5.1) 


2(i-Pjr) 


H y 

-Ai -  (Y  _  n  f  +  —AH 4  _  n 

1  +  MijDj  Vij  j/  1  +  MijDj'  \ij'  j’ 


,2p..t  .  tAiAife 

^  |l  +  M .  .  D . 

\l  1]  J 


/l  +  M.  .D. 
>/  IJ  J 


Define  the  weight  functions 


“ij  * 


M.  . 

-U- 


1  +  M. ,D.  ’ 

ij  J 


i  lj  «  *  •  )N  y  j  — 1  ,  .  »  .  ,  K 


(5.2) 


If  the  prior  means  and  variances,  Hj  ,  D^.  (j=l . k)  are  known,  then,  the 


predictive  quasi-likelihood  function  of  Pjj'  »  8^ven  sample  data  is 


k(Pj  j  'I  Yij  *  ’  i=l»..»»N) 


/,  2  \»n  | 

1 

rpjj’)  exp  j 

0 

2 (1-Pj j  ') 

N 


£  VYu  -  V*  + 


(5.3) 


N 


-nJ') 


A/VV  (YU  V) 


(5.3)  is  called  a  'quasi-likelihood'  since  the  correct  likelihood 
function  is  based  on  the  k-variate  distributions  of  Y . Y„  . 
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In  practice,  and  are  unknown,  we  will  therefore  use  the 

quasi-likelihood  function  of  Pj j '  by  substituting  in  (5.3)  estimates  of 

H.  and  D.  .  Thus,  let 
J  J 


n  M  ^  ~  9 

ST  *  E  w. .(Y. .  -  n.) 

J  i=1  iJ  J 


,  j  1 , • . . ,k 


(5.4) 


p..f  =  E  Jw. .w  ,  (y  .  -  n.)  (y . . ,  -  n.,)  ,  j+j' 

JJ  iJ  iJ  ij  J  ij  J  TJ 


where 


„  N  /N 

3 .  =  E  W..Y.  ./E  W  . 

J  i-1  ij  iJ 


(5.5) 


WiJ  -  V(1+  Vj,N>  * 


(5.6) 


The  estimator  p.  ,  which  maximizes  the  log-quasi-likelihood 
J  >  J 


£(Pjj,) 


N  .  ,,  2 

—  1°8  U-Pjj.)  " 


2p. . ,P  . , 
JJ  jJ 


2  2 

S.  +  ST,- 
J  J’ 


(5.7) 


is  called  quasi-maximum  likelihood  (q.m.l.)  estimator  of  Pj j '  •  The 
derivative  of  £(p)  with  respect  to  p  is 


_|M£i  .  „  — £— 

^  1-p2 


p(s2  +  s2  )  -  p,,,(i  +  p2) 


2\2 
(1-P  ) 


(5.8) 
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Accordingly,  the  q.m.l.  estimator  of  Pj  j  '  *s  a  solution  of  the  cubic 
equation 


3  P11-  2  Pi  A 

p  -  +Hh  +  ~i 


si'  \  pjj- 

- 1  p  -  =  0 

n  /  n 


(5.9) 


for  which  the  second  derivative  of  (5.7)  is  negative.  Accordingly,  for 

A  A 

given  values  of  D.  (j=l,...,k)  ,  we  determine  W..  (i=l . N.  j=l,...,k)  , 

J  > «  1 

and  n j  (j=l,...,k)  and  solve  (5.9)  by  the  Newton-Raphson  method,  starting 
with  the  initial  solutions 


P,  =  2?  ..,/(S2  +  S2,)  ,  j=l . k 

J  .J  J  j  J 


(5.10) 


In  Table  5.1  we  provide  the  initial  estimates  of  the  correlations  (5.10)  and 
the  final  estimates,  which  are  the  solutions  of  (5.9).  The  estimates  of 
Dj  are  those  given  in  Table  4.2.  Two  numbers  are  given  in  each  cell  of  the 

matrix  of  final  estimates  of  the  correlations.  The  upper  number  is  the 
correlation  and  the  lower  number  is  an  estimate  of  its  standard  error, 
namely 


S.E.  {Pjj.}  “  Ci-Pj j 


-P?4,)//n"(1  +  P?  ",)  . 


(5.11) 


Every  estimate  Pjj*  i  which,  in  absolute  value,  is  greater  than  2  S.E.tpjj,} 


is  significant. 


-  12  - 


Table  5.1 


Initial  and  final  estimates  of  the 
correlation  parameters 


INITIAL  ESTIMATES  OF  CORRELATIONS 


1.000 

0.765 

0.168 

0.360 

0.373 

0.678 

0.593 

0.692 

0.616 

0.406 

0.765 

1.000 

0.029 

0.442 

0.444 

0.851 

0.781 

0.763 

0.635 

0.593 

0.168 

0.029 

1.000 

0.321 

0.321 

0.183 

0.122 

0.099 

-0.133 

0.140 

0.360 

0.442 

0.321 

1.000 

0.385 

0.341 

0.336 

0.381 

0.305 

0.565 

0.373 

0.444 

0.321 

0.385 

1.000 

0.613 

0.560 

0.542 

0.364 

0.354 

0.678 

0.851 

0.183 

0.341 

0.613 

1.000 

0.924 

0.721 

0.623 

0.386 

0.593 

0.781 

0.122 

0.336 

0.560 

0.924 

1.000 

0.719 

0.641 

0.333 

0.692 

0.763 

3.099 

0.381 

0.542 

0.721 

0.719 

1.000 

0.491 

0.489 

0.616 

0.635 

-0.133 

0.305 

0.364 

0.623 

0.641 

0.491 

1.000 

0.322 

0.406 

0.593 

0.140 

0.565 

0.354 

0.386 

0.333 

0.489 

0.322 

1.000 

FINAL  ESTIMATES  OF 

CORRELATIONS 

1 

1.000 

0.760 

0.163 

0.355 

0.348 

0.676 

0.587 

0.694 

0.583 

0.394 

0.0 

0.065 

0.185 

0.158 

0.160 

0.087 

0.109 

0.082 

0.110 

0.151 

2 

0.760 

1.000 

0.029 

0.443 

0.421 

0.853 

0.781 

0.769 

0.607 

0.587 

0.065 

0.0 

0.192 

0.141 

0.146 

0.040 

0.059 

0.062 

0.104 

0.109 

3 

0.163 

0.029 

1.000 

0.319 

0.299 

0.183 

0.120 

0.101 

-0.120 

0.136 

0.185 

0.192 

0.0 

0.165 

0.168 

0.183 

0.188 

0.190 

0.188 

0.187 

4 

0.355 

0.443 

0.319 

1.000 

0.366 

0.347 

0.339 

0.392 

0.283 

0.561 

0.158 

0.141 

0.165 

0.0 

0.157 

0.160 

0.161 

0.152 

0.170 

0.115 

5 

0.348 

0.421 

0.299 

0.366 

1.000 

0.596 

0.538 

0.527 

0.322 

0.330 

0.160 

0.146 

0.168 

0.157 

0.0 

0.107 

0.120 

0.123 

0.164 

0.163 

6 

0.676 

0.853 

0.183 

0.347 

0.596 

1.000 

0.926 

0.732 

0.599 

0.385 

0.087 

0.040 

0.183 

0,160 

0.107 

0.0 

0.020 

0.072 

0.106 

0.153 

7 

0.587 

0.781 

0.120 

0.339 

0.538 

0.926 

1.000 

0.726 

0.615 

0.328 

0.109 

0.059 

0.188 

0.161 

0.120 

0.020 

0.0 

0.074 

0.102 

0.163 

8 

0.694 

0.769 

0.101 

0.392 

0.527 

0.732 

0.726 

1.000 

0.469 

0.492 

0.082 

0.062 

0.190 

0.152 

0.123 

0.072 

0.074 

0.0 

0.136 

0.131 

9 

0.583 

0.607 

-0.120 

0.283 

0.322 

0.599 

0.615 

0.469 

1.000 

0.294 

0.110 

0.104 

0.188 

0.170 

0.164 

0.106 

0.102 

0.136 

0.0 

0.169 

10 

0.394 

0.587 

0.136 

0.561 

0.330 

0.385 

0.328 

0.492 

0.294 

1.000 

0.151 

0.109 

0.187 

0.115 

0.163 

0.153 

0.163 

0.131 

0.169 

0.0 
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6.  Summary 

In  the  current  paper  we  present  confidence  intervals  for  the 
category  scores  as  well  as  the  correlations  between  the  scores.  The  cat¬ 
egory  scores  help  determine  the  weaknesses  and  the  strengths  of  the  units. 

It  is  important  to  obtain  confidence  intervals  on  these  scores  because 
before  planning  rt medial  training  it  is  necessary  to  know  whether  or  not 
the  scores  are  meaningful.  Thus  high  correlation  between  two  categories 
may  indicate  that  training  to  improve  the  performance  in  one  can  improve 
the  performance  in  the  other. 

The  results  in  Table  5.1  are  intuitively  explainable.  We  obtain 
high  correlation  between  the  categories  evaluating  the  command's  perfor¬ 
mance  -  Categories  1,  2,  6,  7,  and  8.  But  Categories  3  (Communicating), 

4  (Performing  as  Marines),  5  (Delivering  supporting  fire)  and  10  (Super¬ 
vising  required  actions  of  individual  Marines)  evaluate  the  performance 
of  various  specific  groups  and  thus  each  of  them  cannot  be  expected  to 
have  high  correlation  with  other  categories. 

A  major  finding  -  see  Appendix  A  -  is  that,  for  any  one  battalion, 
the  probability  of  "Yes"  for  a  given  requirement  is  the  same  for  applicable 
requirements  of  the  same  category.  This  provides  considerable  justifica¬ 
tion  for  the  use  of  10  category  scores  as  a  measure  of  the  overall  capability 
of  the  battalion. 
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APPENDIX  1 

We  consider  here  the  question  of  testing  assumption  A. 2. 

TypicaJly,  for  a  given  evaluation,  there  are  a  certain  number  of 
"Yes's"  among  the  applicable  requirements.  The  question  is  how  to  test  the 
hypothesis  that  the  probability  of  a  "Yes"  in  a  given  evaluation  depends 
only  on  the  category  to  which  the  requirement  belongs.  We  have  tested 
whether  this  hypothesis  is  supported  by  the  data,  in  the  following  manner. 

The  data  sets  corresponding  to  the  N  =  27  different  evaluations  are 
arranged  according  to  tasks  within  17  different  Military  Performance  Stan¬ 
dards  (MPS;  see  [7]).  We  therefore  considered  first  the  number  of  "Yes's" 
among  the  applicable  requirements  within  each  cell  of  MPS  x  category,  for 
each  evaluation.  In  Table  A.l  we  present  these  statistics,  as  an  example, 
for  EVAL  1.  An  exact  test  of  the  hypothesis  that  the  probability  of  "Yes" 
is  the  same  for  all  the  MPS's  within  a  category  would  be  very  difficult 
to  perform,  due  to  the  small  number  of  applicable  requirements  in  each 
cell . 

Let  denote  the  probability  of  "Yes"  in  EVAL  i,  MPS  k  , 

Category  j  (i=l,...,27  ;  k=l,...17  ;  j=l,...,10)  .  We  consider  the 
set  of  null  hypotheses: 

0,,  ,  =  Q. .  for  all  k=l,...,17; 

0  ikj  lj  - 

at  each  i=l,...,27  ;  j=l,...,10  . 


In  the  spirit  of  the  Bayesian  approach  of  Section  3,  we  assume,  a  priori, 
that  0^j  are  independent  random  variables,  having  some  prior  distributions 


H,  .(0)  (k=l, . . . ,17  ;  j=l, . . . ,10)  .  If  the  null  hypotheses  are 

true  for  all  (i,j)  ,  the  following  null  hypotheses  should  also  be  true: 
For  each  j  ,  j=l,...,10  , 


Ho  ’j):  \j  =  4  j  l£E  Jill  k=1 . 17  , 

where 

=  /  0ijkdHkj(°i.1k) 

0 


(A. 1.1) 
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Let  ^  kj  3n<^  ^kj  (k=l,...,17  ,  j  =  l, . . .  ,10)  be  the  number  of 

satisfied  and  applicable  requirements  in  the  k-th  MPS  and  j-th  category, 
summed  over  all  the  27  evaluations.  Let 


and 


* 

X  . 
•J 


17 

Z 

k=l 


(J-1.....10) 


* 

M  . 
•J 


17 

Z 


k=l 


(A. 1.2) 


These  statistics  are  given  in  Table  A. 2.  We  see  in  Table  A. 2  that 

ft  ft 

the  values  of  M# .  are  large  ones,  for  all  j=l,...,k  ;  and  that  M 

'  J  kj 

are  generally  not  too  small.  We  therefore  consider,  for  each  hypothesis 
Hq  the  chi-squared  statistic 


TT 

J 


*  *  2 

„  k*  (X.  .  -  E.  .) 

2  _ faL  — 

E* 
kj 


z- 

k=l 


Lkj 


(A. 1.3) 


where 


*  * 

E.  .  =  M, 
kj  kj 


* 

X  . 
‘J 


M 


* 

Xkj  = 


j 

if  M^.  >0 

±f  v° 


k=l, 

j=l, 


,17 

,10 


(A. 1.4) 


(A.1.5) 
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and  k  =  £  I,  .  .  The  estimates  of  the  expected  values  of  X,  .  , 

J  k=1  kj  kj 


under 


the  null  hypothesis  H 


(•  ,j) 
0 


,  are  E,  .  The  large  sample  distribution  of 
kj 


2  * 

T.  is  like  that  of  a  chi-squared  random  variable  with  =  k^  -  1  degrees 

2 

of  freedom.  In  Table  A. 3  we  provide  the  values  of  Tj  and  their  P  -  values 


Pj  =  P  U2lv  J  >  Tj|Tj>  .  j-1 . 10 


(A. 1.6) 
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Table  A. 2 

I 

*  * 

The  statistics  X  .  and  M  . 


3 

i 

214 

158 

66 

366 

l. 

245 

199 

93 

472 

2 

212 

0 

416 

18 

2 

232 

0 

447 

26 

3 

0 

0 

36 

0 

3 

0 

0 

49 

0 

1 

4 

63 

0 

0 

7 

4 

63 

0 

0 

9 

5 

110 

99 

11 

33 

5 

121 

110 

15 

37 

6 

101 

64 

39 

0 

t 

6 

113 

74 

40 

0 

7 

225 

123 

22 

0 

7 

243 

140 

22 

0 

8 

48 

54 

30 

22 

8 

53 

68 

31 

26 

9 

118 

89 

40 

38 

9 

123 

101 

44 

51 

1 

10 

81 

26 

0 

0 

10 

83 

26 

0 

0 

11 

48 

31 

11 

0 

11 

48 

33 

12 

0 

12 

49 

87 

19 

0 

12 

54 

100 

19 

0 

! 

13 

21 

17 

7 

0 

13 

24 

18 

7 

0 

14 

28 

22 

0 

0 

14 

30 

23 

0 

0 

■ 

15 

10 

19 

29 

0 

15 

10 

19 

30 

0 

16 

8 

24 

2 

0 

16 

8 

24 

2 

0 

17 

0 

39 

3 

0 

17 

0 

43 

3 

0 

TOTAL  I 

1336 

852 

731 

484 

r 

TOTAL  N 

1450 

978 

814 

621 

•  J  *1 


0 

65 

351 

216 

64 

219 

0 

70 

413 

283 

75 

258 

162 

218 

367 

204 

207 

70 

182 

239 

440 

228 

225 

75 

993 

77 

21 

0 

0 

0 

1121 

106 

24 

0 

0 

0 

13 

48 

24 

74 

24 

18 

13 

49 

27 

77 

27 

18 

24 

142 

51 

162 

20 

26 

25 

164 

54 

189 

22 

34 

10 

32 

9 

180 

0 

0 

10 

37 

10 

193 

0 

0 

36 

260 

178 

347 

43 

20 

40 

282 

211 

372 

43 

21 

0 

97 

62 

70 

7 

0 

0 

105 

72 

89 

8 

0 

30 

261 

96 

86 

31 

0 

33 

293 

108 

94 

32 

0 

5 

132 

0 

80 

32 

0 

5 

142 

0 

95 

37 

0 

2 

74 

0 

104 

0 

19 

2 

76 

0 

107 

0 

21 

13 

75 

30 

59 

0 

0 

15 

90 

36 

75 

0 

0 

9 

30 

4 

41 

5 

0 

10 

36 

4 

44 

6 

0 

0 

37 

26 

0 

0 

0 

0 

38 

32 

0 

0 

0 

40 

178 

0 

0 

20 

0 

40 

181 

0 

0 

20 

0 

0 

5 

8 

12 

20 

0 

0 

5 

10 

12 

28 

0 

0 

11 

0 

0 

0 

0 

0 

13 

0 

0 

0 

0 

1337 

1742 

1227 

1635 

473 

372 

1496 

1926 

1441 

1858 

523 

427 

r 

I. 

t 
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We  see  in  Table  A. 3  that  each  one  of  the  ten  null-hypotheses 

is  strongly  accepted  (very  high  P  -  values) .  Let  P  be  the  combined 
P  -  value  of  all  the  ten  tests.  According  to  the  Bonferroni  inequality  [4], 

*  10 

P  >  1  -  E  (1-P.)  .  (A. 1.7) 

j=l  3 

: k 

Thus,  according  to  the  values  in  Table  A. 3,  P  >  .30988  .  Thus,  we 

* 

simultaneously  accept  all  the  ten  null  hypotheses,  with  a  very  high  P 
value.  Note  that,  if  at  least  one  of  the  null  hypotheses  hed  been  rejected 
we  should  have  concluded  that  assumption  A. 2  of  Section  2  is  not  supported 
by  the  data.  However,  since  all  the  null  hypotheses  have  been  strongly 
accepted,  we  conclude  that  the  data  is  in  accord  with  assumption  A. 2. 
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APPENDIX  2 


We  show  in  the  present  appendix  that  asymptotically,  the  correlation 

/v  ^ 

between  0„  and  0  „ ,  is  equivalent  to  p  , ,  for  j,j'=l,...,k  . 

The  inverse  of  (2.1)  is 


0..  =  sin2  (Y../2)  ,  i=l,...,N 

13  13  i=l  k 


(A. 2.1) 


where  Y.  .  ^  N  (rj .  ,  ,  M..-1)  .  Expand  0..  around  n .  .  to  obtain 
ij  ij  ij  ij  ij 


®ij  =  sin2(rilj/2)  +  (Y^  -  nij )  sin  (n^/2)  cos  (n±j/2) 


0p(vr  ) 


(A. 2. 2) 


where  0p(-)  is  an  order-of-magnitude-in-probability  symbol  (see  Zacks 
[5;  pp.  208 J )  . 


~  ^  ~  ^  /  O  11  o  Wn  \f 

Let  ®i  =  ^0il  and  %i  =  \sin2( — 2 — )  » •  -  •  fSinM-y^)  )  • 

Notice  that  sin(— ^— )  cos  (-—)  =  — sin(n)  .  Let  (D)  = 


diag  (sin(nil) . sin(nlk)}  . 


Then, 


i  '  h  +  -r  (D)i  -«i>  +  Vj=->  • 


(A. 2. 3) 


Thus,  as  N  ■*  00  ,  the  asymptotic  covariance  matrix  of  0,  is 

■\,i 


t  <V  •  ~r  <»>i  h<D)i  ■ 


(A. 2. 4) 


where  the  elements  of  ^  are  specified  in  (2.2).  Accordingly,  the 


asymptotic  variance  of  0  is 
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